A variety of technologies can be used to investigate biological processes and anatomy. The following examples are types of scan that may be used to provide medical images: X-Ray; Computed Tomography (CT); Ultrasound (US); Magnetic Resonance Imaging (MRI); Single Photon Emission Tomography (SPECT); and Positron Emission Tomography (PET). Each type of scan is referred to as an ‘imaging modality’.
In medical imaging, typically, digital 3-dimensional images are produced. Medical imaging workstations are commonly used to allow the viewing and manipulation of these images. Medical images are read, analysed and reviewed by specialists, for example radiologists.
Typically, a scan provides a ‘dataset’. The dataset comprises digital information about the value of a variable at each of many points. The points are different spatial locations that are spread throughout 3 physical dimensions, i.e. each point is at a particular location on a three dimensional grid. The variable may typically be an intensity measurement. The intensity may be, for example, an indication of the X-Ray attenuation of the tissue at each particular point.
In such a three dimensional dataset, the element of the scan image located at a particular spatial location may be referred to as a ‘voxel’. A voxel is therefore analogous to a ‘pixel’ of a conventional 2-Dimensional image.
Although the dataset of the medical scan is 3-Dimensional, it is typically displayed to a user as a two dimensional image on a medical imaging workstation. An image slice from a 3-d dataset is simply a 2-d representation, consisting of those data points that lie on a particular 2-d plane through the 3-d image. A typical 3-d dataset, such as one from an MRI scan, will have a matrix of regularly spaced data points. As a non-limiting example, the MRI-scan may have data points whose centres are spaced by 1 millimeter in the x- and y-directions across any plane of the scan. Consecutive planes may, for example, be parallel and separated by 7 millimeters.
The 3-D scan may therefore be divided up into tens or hundreds of parallel, 2-D images for display purposes. The user of a workstation can then flick through the images in sequence, for example, thereby allowing a view of successive cross sections of the tissue that was scanned.
Typical workstations allow the 2-D slices to be viewed individually, or sequentially in successive steps. The view may be along a selected one of three perpendicular directions. For a human subject lying down, the axes of the three perpendicular directions may, for example, be along the ‘long axis’ of the body, through the body from top to bottom, and ‘across’ the body from one side to the other. These axes are conventionally referred to as:
(i) ‘axial’, for a cross-section that lies along an axis corresponding to the long axis of the body;
(ii) ‘coronal’, for a cross-section that lies along an axis running from the front to back; and
(iii) ‘sagittal’, for a cross-section that lies along an axis that runs from side to side.
Thus the axial plane is normal to the axis that runs along the head to foot direction, the coronal plane is normal to the front to back axis and sagittal plane is normal to the axis that runs left to right
Henceforth, the term ‘scan image’ should be construed as meaning a three dimensional dataset that results from performing a medical scan. However, when the scan image is displayed, only a two dimensional slice of the dataset may be on view at any one time as an image.
Medical scan images usually have as their subject humans. However, scan images may also be obtained of non-human animals, particularly as part of medical research projects.
Medical scan images may include information about a wide variety of anatomical features and structures. For example, a scan image may show various types of healthy tissue, such as bone and organs within the body. A scan image may also show abnormal tissues. The term ‘lesion’ is often used to describe various types of abnormal tissue. One common example of a lesion is a tumour. However, a scan image may also show other types of lesions, such as cysts or swollen glands. The word ‘lesion’ should henceforth be construed to include both tumours and other types of abnormal tissues.
The purpose of obtaining a medical scan image is often to detect abnormal tissue. So, a typical example of an application of medical imaging is in the identification and ‘staging’ of cancerous tumours.
‘Multiple modalities’ may be used to provide medical scan images. This approach involves obtaining scan images of the same region of tissue by more than one modality. For example, the same region of tissue may be imaged using both a PET scan and a CT scan. Another important example of a multiple mode scan is a SPECT/CT scan. Both PET/CT and SPECT/CT scans combine the predominantly anatomical and structural information obtained from a CT scan with a scan which measures the biological function.
Scanners that can carry out multiple mode scans are referred to as ‘hybrid scanners’. Typically, a hybrid scanner allows the subject to be scanned by both modalities in the same sitting.
A key task in the interpretation of medical image scans is the need for a user to be able to define a region on a scan image. That region is henceforth referred to as a ‘region of interest’. A typical region of interest is a portion of a scan that shows a particular anatomical feature or structure. However, this leads to the task of defining how one region of interest in a first scan image relates to corresponding portions of other scan images with the same content, e.g. images of the same patient. This task arises in particular in connection with radiotherapy. The process of defining a region is often referred to as ‘contouring’.
Working with Multiple Scan Images in Radiotherapy
In radiotherapy, the aim is to deliver a high radiation dose to cancerous tissues. Simultaneously, the radiation given to nearby normal tissues must be minimised. One approach to achieving this is ‘image based planning’ In simple terms, this means planning radiation dosage, partly on the basis of information visible in medical scan images. An image of a patient, i.e. a dataset, may be available that is from a 2D, 3D or even a 4D medical scan.
The first step of image based planning is to define contours or regions of interest on a single planning image. The planning image is normally a CT image. The contours or regions of interest delineate:
(i) the location of target regions, e.g. the tumour, for treatment; and
(ii) normal structures for avoidance.
One planning image may in fact have several regions of interest, each defined by a separate set of contours.
A process known as simulation is then used to estimate the radiation that should be delivered to the structures, for a given treatment plan.
Contours or regions of interest may also be defined in medical images for reasons other than radiation treatment planning. One example is for accurate lesion measurement during diagnosis.
Multiple clinical images for a patient may be acquired for using a variety of modalities, such as CT, PET and MR. Each of these images provides anatomical and functional information, at different resolutions. Each type of image brings some advantages, for example MR images show good soft-tissue discrimination, which may enable the identification of the boundaries of a tumour. A CT image is typically used for radiotherapy contouring, because the attenuation of x-rays can be better estimated using CT within the treatment simulation and planning.
Therefore it may be beneficial to use multiple images and image modalities within the contouring process, to enable more precise contouring. See reference [1] at the end of this background section.
The usual prior art approach to using multiple images and image modalities within the contouring process involves aligning the multiple images to each other. In this approach, the images are first aligned to a common frame of reference. This alignment is done in such a way that contours defined on one image may be transferred directly to another. How this is done depends on the particular group of images that are available.
In the simplest case, images may have been acquired in the same imaging study, on the same scanner. That is, the multiple images were acquired either sequentially or concurrently, while the patient remains stationary in the scanner. Such images are referred to as being in the same ‘frame of reference’. An example is when multiple sequences of MRs are acquired sequentially, of one patient at ‘one sitting’.
In some situations, it is not possible to acquire all the images in the same frame of reference, i.e. in the same scanner. For example, different scanner types might be necessary.
FIG. 1 shows an example of two images. In this example, the images have been obtained of the same patient, but at different times. In addition, different scanners have been used. The medical scan images of FIG. 1 may be displayed by a medical imaging workstation.
Reference 110 shows the screen of the medical imaging workstation. The result of a first scan is shown as first scan image 120. Adjacent to first scan image 120 is the result of a second scan, which is second scan image 130. A portion of tissue generally labelled 140 is shown on first scan image 140. At a slightly different scale, the same portion of tissue is shown and labelled 150 on second scan image 130. The Region of Interest ‘ROI’ differs in shape and size between the two images. This difference may arise due to one or more of: the different scale of the two scans; movement of the patient between the two scans; and the different display orientations of the two displayed images.
The usual prior art approach to images that are not in the same frame of reference is to aligned the images using a more complex transformation than was needed for images that are in the same frame of reference. This process of aligning images is known as ‘image registration’.
The primary aim of image registration in contour planning is simply to correct for differences in patient position.
There are three well known image registration methods. These are termed ‘rigid’, ‘affine’ and ‘deformable’ registration. FIGS. 2-4 illustrate each of these registration methods. FIGS. 2-4 are shown on a single page, in order to facilitate comparison between the three approaches.
FIG. 2 shows a rigid alignment method of image registration. In the example of FIG. 2, in three dimensions (3-D), 6 parameters require calculation: Translations (3 parameters); Rotation (3 parameters).
FIG. 3 shows an affine alignment method of image registration. In the example of FIG. 3, in 3-D, 12 parameters are required: Translations (3 parameters); Rotation, Shearing and Scale (9 parameters).
FIG. 4 shows a deformable alignment method of image registration. In the example of FIG. 4, in 3-D, 3 parameters are required per image element: Translations (3 parameters) at each image location. This can be thousands of parameters for a 3D image.
References [2] and [3] explain other methods of aligning pairs of images.
There are a number of techniques in the prior art which allow a user to delineate regions using multiple imaging volumes. One approach presents a first image as a base layer, over which one or more semi-transparent overlays are displayed. Each semi-transparent image is derived from a different image than the first image. This approach is known as a ‘fused view’ in medical imaging, see ref [1]. Here, objects are drawn by the user, and these objects are created in the geometric space of the first image. These objects will thus be shown in the base layer. This enables the user to define the contour on the first image, whilst being able to view and use information from the overlying images.
However, the various images may be acquired at different orientations and resolutions. So either a rigid or non-rigid transformation is usually required to produce each overlay image. As a consequence, the image data shown to the user in the overlay images(s) is not the originally captured image data for that image. The data has been warped or rotated, or in some other way resampled, in order to create the overlay image.
This may be problematic for several reasons:
(i) The resolution of the image shown in the overlay may not produce resampled images of sufficient quality. For example, MR images are typically highly anisotropic, which means that the voxels may not be cuboid. The voxels may typically be 3 mm×0.3 mm×8 mm. Such images are best viewed in their original orientation, and do not produce clear images if rotated or warped. The application of translations and isotropic scalings are acceptable, however. These are needed for zooming and panning operations that are useful in medical imaging visualisation software.(ii) Aside from visual quality, transformation of the overlay image also means that its voxels are modified. This modification means that the voxels are no longer the original ones acquired from the scanner. In some circumstances, it is beneficial to be able to use the originally acquired voxels. One example is in PET, where the values of the voxels convey information that it is important to preserve.
Another method in the prior art involves warping a region of interest to multiple different images. Such an approach may begin with one or more regions that were defined on a previous planning image. These regions are then transformed to a new planning image, for example in order to start planning a new phase of radiotherapy. This process is known as ‘re-planning’. See reference [4]. Re-planning is applied where, for example, a patient is being retreated for recurring disease. The user then warps the previously defined regions of interest from the previous planning volume to the new one.
In this approach, each region of interest is resampled into the space (frame of reference) of the new planning volume, using a transformation that maps one volume onto the other. As discussed earlier, a registration algorithm may be used to estimate such a transformation between the previous and the new planning images. However, in this case, a new region of interest is simply created on the new planning volume. The object of this approach is simply to create the new region on the new planning image, whilst attempting to avoid losing any information that could still be useful from the region of interest that was originally created on the original planning image. Critically, such re-planning systems are not designed to help the user define one region using multiple images and have limited functionality, for example:
(i) Any such regions are treated as two separate and unrelated ‘objects’ by the system. A user would be required to warp the region of interest from the new planning image back to the original planning volume, if they wanted to make further edits using information on that image. After any edits, the region would again have to be warped back to the new volume to continue with the replanning operation.(ii) Prior art re-planning systems are designed to allow the user to edit regions in just two images at a time, which are usually spaced over a period of days, weeks or months.
The concept of using multiples images for automated segmentation has been considered in academic research. See the approach in reference [5]. However, such approaches have the drawback that the user has no control, and cannot adapt the process.
FIG. 5 shows an illustration of the extent of a region of interest in a scan image. This may be accomplished in one of several ways. Two of these are as follows:
(i) Using a ‘boundary box’. The boundary box is a 3-Dimensional shape, for example a cuboid or an ellipsoid. A user may define the boundary box, based on what can be seen on the first scan. A cursor on a screen of a medical imaging workstation may be used to define the boundary box, under the control of a mouse or tracking ball. The boundary box is typically placed so as to encompass all of an object that is to be analysed. A threshold can then be set. The ‘first region of interest’ then comprises all the spatial locations within the boundary box at which the measured value exceeds the threshold.(ii) A variant of approach (i) is to define a boundary box, and then find the maximum value of any spatial location within the boundary box. A percentage of the maximum value, for example 40% of the maximum intensity, is then selected as a threshold. The ‘first region of interest’ then comprises all the spatial locations within the boundary box at which the measured value exceeds the threshold.FIG. 5 shows an example of a boundary box 210 that may be used in approaches (i) or (ii) above. Within boundary box 510 is an area of tissue that the operator of a medical imaging workstation suspects may be a lesion. In accordance with the approach outlined under (i) above, a medical imaging workstation or the hybrid scanner identifies all the spatial locations within boundary box 510 where a threshold value, for example of intensity, is exceeded. These locations form the first Region of Interest ROI.